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Abstract 

We study (backward) stochastic differential equations with noise com- 
ing from a finite state Markov chain. We show that, for the solutions of 
these equations to be 'Markovian', in the sense that they are determin- 
istic functions of the state of the underlying chain, the integrand must 
be of a specific form. This allows us to connect these equations to cou- 
pled systems of ODEs, and hence to give fast numerical methods for the 
evaluation of Markov-Chain BSDEs. 
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1 Introduction 

Over the past 20 years, the role of stochastic methods in control has been 
increasing. In particular, the theory of Backward Stochastic Differential Equa- 
tions, initiated by Pardoux and Peng [5], has shown itself to be a useful tool 
for the analysis of a variety of stochastic control problems (see, for example, El 
Karoui, Peng and Quenez [6] for a review of applications in finance, or Yong and 
Zhou [TT] for a more general control perspective) . Recent work [U [3] has consid- 
ered these equations where noise is generated by a continuous-time finite-state 
Markov Chain, rather than by a Brownian motion. 

Applications of BSDEs frequently depend on the ability to compute solu- 
tions to these equations numerically. While part of the power of the theory of 
BSDEs is its ability to deal with non-Markovian control problems, the numerical 
methods that have been developed are typically still restricted to the Markovian 
case (see, for example, [21 1])- In this paper, we ask the question 

When does a (B)SDE with underlying noise from a Markov Chain 
admit a 'Markovian' solution, that is, one which can be written as 
a deterministic function of the current state of the chain? 

As we shall see, such a property implies strong restrictions on the parameters of 
the (B)SDE. However, these restrictions form a type of nonlinear Feynman-Kac 
result, connecting solutions of these SDEs to solutions of coupled systems of 
ODEs. This connection yields simple methods of obtaining numerical solutions 
to a wide class of BSDEs in this context. 



2 Markov Chains and SDEs 



2.1 Martingales and Markov Chains 

Consider a continuous-time finite-state Markov chain X on a probability space 
(fi, P). (The case where X is a countable state process can also be treated in this 
manner, we exclude it only for technical simplicity.) Without loss of generality, 
we shall represent X as taking values from the standard basis vectors e$ of M. N , 
where N is the number of states. An element lu £ f2 can be thought of as 
describing a path of the chain X. 

Let {Tt} be the completion of the filtration generated by X, that is, 

F t = a({X s } s < t ) V{4eJ: P(A) = 0}. 

As X is a right-continuous pure jump process which does not jump at time 0, 
this filtration is right-continuous. We assume that X is deterministic, so J-"o is 
the completion of the trivial a-algebra. 

Let A denote the rate matrisfHof the chain X. As we do not assume time- 
homogeneity, A is permitted to vary (deterministically) through time. We shall 
assume for simplicity that the rate of jumping from any state is bounded, that 
is, all components of A are uniformly bounded in time. Note that (A t )ij > 
for i 7^ j and ^ Aij = for all j (the columns of A all sum to 0). 

It will also be convenient to assume that V(X t = e,) > for any t > and 
any basis vector £ M. , that is, there is instant access from our starting state 
to any other state of the chain. None of our results depend on this assumption in 
any significant way, however without it, we shall be constantly forced to specify 
very peculiar null-sets, for states which cannot be accessed before time t. If we 
were to assume time- homogeneity (that is, A is constant in t), this assumption 
would simply be that our chain is irreducible. However, this assumption does 
not mean that Ay > for all i ^ j. 

From a notational perspective, as e, denotes the ith standard basis vector 
in HL N , the ith component of a vector v is written e*v, where [•]* denotes vector 
transposition. For example, this implies that useful quantities can be written 
simply in terms of vector products. For example, we have Ix t =ei = e *^t- 

2.2 Markov-Chain SDEs 

We now relate our Markov chain to a A^-dimensional martingale process, with 
which we can study SDEs. To do this, we write our chain in the following way 

X t = X + / A u X u _du + Alt 
J)o,t] 

where M is a locally-finite- variation pure-jump martingale in H. . Our attention 
is then on the properties of stochastic integrals with respect to M. 

We shall make some use of the following seminorm, which arises from the 
Ito isometry. 

our notation, A is the matrix with entries Aij, where Ay is the rate of jumping from 
state j to state i. Depending on the convention used, this is either the rate matrix or its 
transpose. 
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Definition 1. Let Z be a vector ir 



»JV 



Define the stochastic seminorm 



\Z\ 



Mt 



Ty[z ,d { M,M) tz 



where Tr denotes the trace and 

d(M, M) t 
dt 



&mg(A t X t _) - A t diag(AV) - diag(X t -K =: *(A t ,X t ) (1) 



the matrix of derivatives of the quadratic covariation matrix of M . This 
seminorm has the property that 



E 



\ z u\\ M du 



E 




Z*dAL. 



for any predictable process Z of appropriate dimension. We define the equiva- 
lence relation ~ M on the space of predictable processes by Z ~ M Z' if and only 
if \\Z t - Z[\\ Mt = dtx dV-a.s. 

Remark 1 . A consequence of this choice of seminorm is that Z + cl ~ M Z for 
any Z and any predictable scalar process c. This is simply because e i^- e j — 
J2i Aij — for all j, and so all row and column sums of ^(A t ,X t ) are zero. 

Theorem 1. Every scalar square-integrable martingale L can be written in the 
form 



Lt — Lq 



Z*dNh 



o./.] 



for some predictable process Z taking values in 
to equivalence ~m- 



The process Z is unique up 



Proof. See 0]. 

The key SDEs which we shall study are equations of the form 
dY t = -f(u, t, Y t -,Z t )dt + Z* t dM t 



□ 



(2) 



where / : ft x 



is a progressively measurable function, Y 



is an adapted process with E[sup t<T Y t ] < oo for all T and Z is predictable. 
A solution Y to this equation is a special semimartingale, hence the canonical 
decomposition into a predictable part (— J f{uj,t,Yt-,Z t )dt) and a martingale 
part (J Z*dMt) is unique. 

We shall make the general assumption that f(u, t, y, •) is invariant with re- 
spect to equivalence ~m> that is, 

if Z ~ M Z' then f(u, t, y, Z) = f(u, t, y, Z') 

up to indistinguishability. This assumption is important, as it ensures that / 
only considers Z in the same way as it affects the integral f Z*dM. 

In terms of existence and uniqueness of solutions to (pi) we shall focus on 
two key cases, 

• first, when Yq € R and {Z t }t>o a predictable process are given, and so ^ 
is a forward SDE, and 
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• second, when Yt <E L?{Tt) is given for some T > 0, and Z t is chosen to 
solve the backward SDE, that is, to ensure that Y is adapted and Z is 
predictable. 

For the forward equation, the existence and uniqueness of solution process Y is 
classical, under various assumptions on the driver /. For the backward equa- 
tion, under the assumption of Lipschitz continuity of the driver /, a result on 
existence and uniqueness of the pair (Y, Z) is given in [3] (where Z is unique 
up to equivalence ~m, as defined in the following definition). In this paper, we 
shall not focus on determining conditions on / such that existence and unique- 
ness of solutions holds, but shall always assume that sufficient conditions are 
placed on / such that the equation of interest has a unique solution. 

Lemma 1. For any pair (Y, Z) satisfying ®, the pair (Y, Z') also satisfies 
if and only if Z ~ M Z' . 

Proof. By the canonical decomposition of Y into a finite variation and a mar- 
tingale part, we see that L Q ( , Z*dM = f^ Q ( , Z'*dM up to indistinguishability, 
and so Z Z' . Conversely, if Z ~ M Z' , then by our assumption on /, (Y, Z') 
will also satisfy ([2]), up to indistinguishability. □ 

Lemma 2. For any martingale L, let AL denote the jumps of L. If Z is 
a predictable process such that AL t — Z t AM t up to indistinguishability, then 
L t — Lq + Jj t j Z*dM u , and Z is unique up to equivalence 

Proof. Let L have a representation dL t = Z' t dM t for some predictable process 
Z'. Then as AL t = Z t AM t up to indistinguishability, we must have that 
B = {(u),t) : \\Z t — Z' t \\M t 7^ 0} is a predictable set such that AM t — on 
B. One can verify that a predictable set with this property is the union of a 
dt x tflP-null set and a set on which A t X t - = 0. However, if A t X t - — then 
d(M, M) /dt = 0, and so we see that Z ~ M Z' . As the martingale representation 
is unique up to equivalence ~mj we have our result. □ 

3 Markovian solutions to (B)SDEs 

Definition 2. We say a stochastic process Y is Markovian if up to indistin- 
guishability, Y t depends on ui only as a function of X t , that is, it can be written 
as 

Y t = u(t,X t ) 
for some deterministic function u. 

We note that this is, in some ways, a misnomer, as a process satisfying Defi- 
nition [2] need not be Markovian in the sense that Y t is conditionally independent 
of Y r given Y s , for all r < s < t. Nevertheless, this terminology is standard in 
the theory of BSDEs, and describes adequately the property of interest. 

Our key result on the structure of Markovian solutions to ([2| is the following. 

Theorem 2. Suppose Y has dynamics given by Q). Then the following state- 
ments are equivalent: 

1. Y is of the form Y t — u(t, X t ) for some function u : ]R + x M. N R. 
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2. Both 

(i) f is Markovian, that is, it is of the form 

f(u,t,Y t ,Z t ) = f{X t ,t,Y u Z t ) P x dt-a.e. 

for some function f : R N x R+ x E x R N -> R and 

(ii) up to ~ m equivalence, Z t is a deterministic function oft which sat- 
isfies Xq Z = Y and for all i 

= -f(e h t,e*Z t ,Z t ) - z;A t e t . 

Furthermore, under either set of conditions, e*Z t — it(t, ej) up to equivalence 

Proof. 1 implies 2. From pj), Y is continuous except at a jump of X. By 
boundedness of A, for any bounded interval [a, b], with positive probability there 
will not be a jump of X in [a, b]. Therefore, for each i, considering the non-null 
set {lo : X t = for all t € [a, b]} we see that it(-,ej) must be continuous on 
every bounded interval [a, b}. 

Consider a jump of X which occurs at the stopping time r. Let L t := 
Jg Z*dM u , that is L is the martingale part of Y. Then from (JsJ) 

al t = y T - y r _ = z;AM t = z*{x T - x T _). 

Define the process i? t ' with components 

4 z 't = u(t,ei). 

Then we have (Z' T )*X T = Y T and (Z' T )*X T - = Y T _, for every jump of X. Hence 
Z' is a predictable (indeed, deterministic) process such that AL — Z' AM, and 
by Lemma [2] we see Z ~ M Z'. 

Now note that, except on the thin set {AX t 7^ 0}, 

^-u(t,X t ) = ^ = -/(a;,i,r t ,Z t )-2?^ t _. 
ere at 

Considering these dynamics on the non-null set {a; : X t — &i for all t G [a, 6]}, 
we obtain 

~u(t,e i ) = -f(u,t,Y t ,Z t )-Z;A t e i . (3) 
As Z t is deterministic, by rearrangement, we see that 

f(-,t,Y t ,Z t ) = -ju{t,X t ) - Z* t A t X t . 

does not vary with ui on the sets where X t is constant. Therefore, we can write 
f(X t ,t,Y t ,Z t ) as the common value taken by / on these sets. Finally, we see 
that replacing / by / and using the fact that e*Z t — u(t, 6j) in ^ yields the 
desired dynamics for Z. 

2 implies 1. By uniqueness of solutions to ([2]), as Yq — X^Zq and Z has the 
prescribed dynamics, we know that Y t — X^Z t up to the first jump of X. At 
the first jump time r, we see that 

Y T — Y T _ = AY T = Z*(yX T — X T —) = Z*X T — Y T —, 
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and so Y T = X* Z T almost surely. Repeating this argument and using induction 
on the (almost surely countable) sequence of jumps of X, we see that Y t — 
X£Z t up to indistinguishability. However Z t is deterministic, so we can define 
a deterministic function u(t, e 2 ) = e*Z t , and we see that Y t = u(t, X t ). □ 

The following corollary provides a frequently more convenient way to analyse 
such equations. 

Corollary 1. Let Y be as in Theorem^ with associated function u. Write 
u t for the column vector with elements u(t,ei). Write f(t, u) for the column 
vector with elements e*f(<,u t ) := /(e^, t, e*u t , u t ). Then u satisfies the vector 
ordinary differential equation 

du t = -(t(t,u t ) + A*u t )dt (4) 

Proof. Simply note that u = Z in the proof of the theorem, and so the dynamics 
are as given. □ 

Corollary 2. Let Y be the solution to a BSDE with Markovian terminal con- 
dition Yt — 4>(Xt), for some deterministic function <f> : M. N — > R. Suppose f is 
Markovian. Then for all t < T ' , Y t — u(t, X t ), where 

. u(T, •)=#•), 

• the associated vector u t satisfies the ODE Q) and 

• the solution process Z is given by Z t = u t . In particular, note that Z is 
deterministic and continuous. 

Proof. Simply define u t as the solution to the ODE Q, working backwards in 
time, with initial value e*u-r = <fi(ei). Then the pair of processs (Y t ,Z t ) := 
(X t *u t ,u f ) is a solution to the BSDE with dynamics ([2]), and hence is unique 
(up to equivalence ~m for Z). The remaining properties follow directly from 
the theorem. □ 



3.1 Brownian BSDE and semilinear PDE 

It is worth comparing these results with those for BSDEs driven by Brownian 
motion. In the simplest classical case, suppose the filtration is generated by a 
scalar Brownian motion W, and consider a BSDE of the form 

dY t = -f(W t , t, Y t , Z t ) + Z t dW t - Y T - <j>{W T ) 

where <f> : K — » R is continuous. 

Then, one can show from the Feynman-Kac theorem (see, for example, [S] 
or [TU]) that Y t = u(t, Wt), where u is the viscosity solution to the semilinear 
PDE ' 

d t u(t, x) = ~f(x, t, u(t, x),d x u(t, x)) ~ -dl x (u(t, x)), u(T, x) = 4>{x). 

Comparing this to our equation Q for u t , we see that first difference is that 
we have replaced the infinitesimal generator of the Brownian motion (^d xx ), by 
the infinitesimal generator of the Markov chain (At). 



G 



The second difference is that, in the Brownian case, f(x, t, •, •) depends only 
on the behaviour of u in a neighbourhood of x. In the Markov chain case, as our 
state space does not have a nice topological structure, it would seem that e*f 
can depend on all the values of u, not only on those 'close' to the ith coordinate 
of u. 

However, when we examine e*f(t,u t ) = /(e^, t, e*u t , u t ), we see that e*f 
depends only on the ith coordinate of u, and on those properties of u which are 
invariant up to equivalence ~m- In particular, if a jump from state to state 
ej is not possible, then e*f cannot depend on the value of e*u t (as changing 
this value will give a vector which is equivalent to u t up to equivalence 
Similarly, if a constant is added to every element of u t , e*f will change only 
through the dependence on e*u t , rather than through any other element of the 
vector. In this sense, the 'local' dependence is preserved by the equivalence 
relation. 

An alternative way of thinking through the relationship between our result 
and those known in the Brownian case is through the following diagram, indi- 
cating how an equation of one type can be converted into another. 

Brownian (B)SDE y > Parabolic (semilinear) PDE 



Space Discretisation 



Finite Element Method 



Markov chain (B)SDE W > Coupled ODE system 

Our result provides the link indicated by (*). It is natural to think that, given 
appropriate choices of spatial discretisations and finite element methods, this 
diagram will commute. 



4 Calculating BSDE solutions 

As we have shown that there is a connection between BSDEs driven by Markov 
chains and systems of ODEs, it is natural to use this connection for the purposes 
of computation. As with classical BSDE, this connection can be exploited in 
both directions, depending on the problem at hand. 

As mentioned before, various practical problems can be analysed using the 
framework of BSDE. Of particular interest are dynamic risk measures and non- 
linear pricing systems, as described in [5]. By connecting this theory with the 
theory of ODEs, we gain access to the large number of tools available for the 
numerical calculation of ODE solutions, the only concern being the dimension- 
ality of the problem. We note that while we shall consider an example from 
finance, the same methods can be applied in other areas of stochastic control. 

We give a practical example taken from Madarj^J Pistorius and Schoutens 
[8j. In this paper a 1600-state Markov chain on a non-uniform spatial grid is 
created to match the behaviour of a stock price in discrete time, assuming that 
there exists an underlying Variance-Gamma local Levy process, based on the 
CGMY model of The transition probabilities are fitted in discrete time 
from month to month, yielding a calibrated discrete time risk-neutral transition 
matrix. 

2 Thanks to Dilip Madan for kindly providing us with access to the fitted data from this 
paper. 
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From this matrix, we extract a continuous-time Markov chain approxima- 
tion, using a carefully constructed approximation of the matrix logarithm. (An 
approximation is needed as the matrix in question is large, and possibly due 
simply to calibration error, does not exactly correspond to the skeleton matrix 
of a continuous time Markov chain. We hope to give the details of this approx- 
imation in future work.) For our purposes, the only relevant quantities are the 
grid used, (that is, the value of the underlying stock in each state), and the rate 
matrix of the Markov chain. 

Let S(X t ) denote the value of the stock in state X t . We shall consider the 
risk-averse valuation of a contingent claim using our BSDE. To do this, we fix 
the terminal value as a function of the state Yt = 4>(S(Xt)) for some function 
4>. We then take the risk-neutral valuation E[Yt], and dynamically perturb this 
through the use of a BSDE with concave driver (if the driver were / = 0, then 
we would simply obtain the risk- neutral price E[Yt]). This yields a process 
Yt = u(t, Xt), which we interpret as the ask price (that is, the amount an agent 
is willing to pay, at time t in state X t ) of the terminal claim Yp. 

The value Y t can be thought of as containing both the risk-neutral price and 
a correction due to risk-aversion. Through the use of a BSDE, we ensure that 
this correction can be dynamically updated, and so our prices are consistent 
through time (they do not admit arbitrage, see [5]). 

If we think of Y t as the ask price, that is, the amount an agent is willing to 
pay to purchase Yr, then it is natural to also ask how much he would be willing 
to sell Yr for, that is, the bid price. This value corresponds to the negative of 
the solution of the BSDE with terminal value —Yr (as selling Yp is equivalent to 
purchasing —Yr, and we change the sign of the final solution so that it represents 
an inward rather than outward cashflow). 

To solve this equation, we then convert our BSDE with driver / and terminal 
value c/>(S(Xt)) into a coupled system of ODEs, using Theorem [2j It is then 
a simple exercise to use any standard ODE toolbox (for our examples we have 
used the ode45 IVP solver in Matlab) to solve the relevant ODE system. 

Example 1. Consider the BSDE with driver 

f(X t ,t,z)= min \r{z*A t X t )}. 

r£ [a~ 1 ,a] 

This equation corresponds to uncertainty about the overall rate of jumping from 
the current state - the parameter a > 1 determines the scale of the uncertainty. 
The uncertainty is, however, only about the overall scale of the jump rate - the 
relative rates of jumping into different states remain the same. 

Note that this driver is concave, and satisfies the requirements of the compar- 
ison theorem in [5]. Hence the solutions to this BSDE give a 'concave nonlinear 
expectation' £{Q\F t ) := Y t in the terminology of [5]- The driver is also posi- 
tively homogenous (that is, f(X t ,t, \z) = Xf(X t ,t, z) for all A > 0) and so the 
nonlinear expectation is positively homogenous, that is, it does not depend on 
the units of measurement. 

For our numerical example, we use a timeframe of one month, set a = 1.1, 
and calculate the value of a simple Butterfly spread 

!s ,se[15,20[ 
25 -s se[20,25[ 
otherwise. 
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Figure 1: Ask price surface for a butterfly spread under rate uncertainty. 
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Figure 2: Time t — bid (high) and ask (low) prices for a butterfly spread 
under rate uncertainty. 

The results of solving the resultant system of ODEs can be seen in Figures [T] 
and [2] In Figure [l] we plot the surface generated by plotting the solution of the 
ith term of the ODE against the stock price S(ei). The assymetry that can be 
seen in Figure [l] is due to assymetry in the rate matrix of the markov chain - the 
volatility of the stock depends on its current level. The prices look qualitatively 
similar to what one would obtain using the classic expectation (which, in fact 
would lie between the bid and ask curves in Figure [2]). As one would hope, the 
bid price lies above the ask price, however we can see that the difference (the 
bid- ask spread) is not constant, and is higher when the stock price is larger. 

An aspect of these prices which is not immediately apparent from the figures 
is that they are dynamically consistent, that is, if prices evolve in this manner, 
then one cannot make an arbitrage profit. This follows from the fact that these 
prices satisfy a BSDE, see [5]- 
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Example 2. In 8], various prices are determined using the estimated discrete 
time model and a nonlinear pricing rule. The technique used is to apply a 
concave distortion the the cumulative distribution function of the values one- 
step ahead. That is, the value at time of a payoff at time 1 with cdf F is given 

by 

xd(l - (1 -F(a;)^) (1+7) ) 



for some 'stress level' 7 > 0. This is called the minmaxvar distortion. 
In the same vein, we now consider the use of the driver 

f(X t ,t,z) = z*(A*-A t )X t ; 

where A\ arises from the continuous time analogue of minmaxvar, where we 
distort the relative rates of jumps to each of the non-current states. This is 
defined by the following algorithm: 

Minmaxvar rate matrix distortion 

1. Sort the components of Zi to give an increasing sequence z^u\, where 7r is 
a permutation of {1, N}. 

2. Define the cumulative sum of the corresponding sorted rates (excluding 
the current state), G(i) = Y! J=1 {el U) A t X t )+ . 

3. Apply a concave distortion to the scaled cumulative sum 

WW = (1 - (1 - ( oit-m ) ) 1 {G{N) ~ G(1)) + G(1) ' 

4. Define the individual distorted rates = ip(G(i)) — ip(G(i — 1)), with 
initial term qi = G(l); 

5. Unsort these rates to define a distorted rate matrix Af 

e* i A z t X t = q 7t -i {i) , {e^Xt} 
with the diagonal then chosen to give row sums of zero. 

We apply this pricing mechanism to a digital option with a knockout barrier, 
that is, a payoff <f> where 

if S(X t ) is ever above 25 

1 if S(X T ) > 15 and S(X t ) < 25 for all t 
otherwise 

This type of simple barrier option is straightforward to calculate, as we simply 
ensure that our solution satisfies the additional boundary condition u(t, X t ) = 
for all states where S(X t ) > 25. Our solution is then the value of the option, 
conditional on the barrier not having been hit. The results of this, using a stress 
level 7 = 0.1, can be seen in Figures [3] and [4] 

Again we can see the effects of risk aversion in the differences of the bid and 
ask prices in Figure [4] The effects of the knockout barrier are also clear, as it 
causes a sharp change in the prices at the boundary. 
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5 Calculating ODE solutions 



It is well known that a potential application of the theory of Brownian BSDEs 
is to provide stochastic methods for large PDE systems. In our situation, we 
have seen that the natural relation is not between a BSDE and a PDE, but a 
BSDE and a system of coupled ODE. We therefore can consider adaptations of 
the stochastic methods for solving BSDE as candidates for novel schemes for 
solving large systems of coupled nonlinear ODE. As numerical algorithms for 
ODEs are generally very good, we do not expect that this method will be of use 
except in some extreme cases. For this reason, we simply outline the algorithm. 

To implement such a method, consider the following general setting. Suppose 
we have a system of ODEs of the form 

dv t = g(t,v t )dt; v = 0, (5) 

and the object of particular interest is the value of e£v<r, that is, the value of 
the fcth component of v at some future time T. (This method can, of course, 
be modified to give all components, however is particularly well suited to when 
our interest is in a single component.) 
We split the ODE ^ into the form 

dvt = (f(t,v t )+i4Jvi)tft (6) 

where A t is a rate matrix (that is, a matrix with nonnegative entries off the 
main diagonal and all row sums equal to zero) and f is a function with the 
property that there exists c such that 

|e*(/(i,v t )-/(f,v;))i 2 < c (v f -v;)**(i t ,c ! )(v ( -v;) 

for all i, where •) is as in Practically, this means that the zth component 
of / can only depend on the jth component of v when (A t )ij > 0. 

We then reverse time, defining u t = v*r-t, so that (|6| becomes precisely the 
ODE we obtain from our BSDE @. Therefore, we have converted our problem 
into determining the initial value of a BSDE with terminal value X^<j), when X 
is a Markov chain with rate matrix A t . Furthermore, as our interest is in the 
value of the fcth component of v, we are interested in the initial value of our 
BSDE when X = e k . 

We now outline the natural modification of the algorithm of Bouchard and 
Touzi [2] for the calculation of solutions to these BSDEs. We shall not give a 
proof of the convergence of this algorithm, however it is natural to believe that 
the conditions for convergence from the Brownian setting will carry over to the 
setting of Markov Chains. 

Monte-Carlo Algorithm for Markov Chain BSDE 

First fix a discretisation level At = T/M. 

1. (Forward step) Simulate a large number paths of the Markov chain be- 
ginning in state X = e^, on the discretised grid {kAt}k<M- Write X™ 
for the value of the nth simulation of the Markov chain at time t. The 
method of simulating this forward step can be chosen for the sake of com- 
putational convenience. The cost of doing this will depend on the size of 
the components of the matrix A t . 
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2. (Backward step) Iterate backwards from T to in steps of size At, at 
each step, we seek to create an estimate of the function Ut = u(t, •) : 
{1,2, ...,iV} — > K based on the function u t +i (where N is the number of 
ODEs). 

(a) Use the function u t +i to calculate the values of f n := (X™ +1 )* f(t, u t +i) 
(recall that as X™ +1 takes values from the standard basis vectors of 
M. N , this simply selects out some terms of the function /. 

(b) Calculate 

We wish to approximate 

u t (x) « E[u t+1 (X t+ i)\X t = x] 

This can be done using a Longstaff-Schwarz technique (see [7j), using 
an appropriate basis {V'm} for functions on {1, 2, N}. To do this, 
we choose coefficients c m to minimise 

n \ m / 

This gives the least-squares approximation 

U t (x) = y^X c m<Pm(x)). 

m 

(c) Repeat until an estimate for uq is obtained. 

We note that the accuracy and cost of the backward step primarily de- 
pends on the size and computational cost of evaluating /. 

This algorithm is particularly well suited for calculating the values ito(efc) = 
e£.VT, as the simulated paths can be chosen to start in state e&. Hence the 
algorithm will not attempt to accurately calculate values of Ut{ey) — e^^T-t 
for combinations oft and k' which have little impact on ejlvy. We also note that 
there is a trade off between the costs of the forward and backward processes, 
depending on the decomposition from ^ to Typically, it seems that the 
forward process is relatively cheap to simulate accurately, so it is preferable to 
try and minimise the size of /. 

6 Conclusion 

We have considered solutions to Backward Stochastic Differential Equations 
where noise is generated by a Markov chain. We have seen that solutions to 
these equations are markovian, that is, they can be expressed as a deterministic 
function of the current state, if and only if they come from the solution of a 
certain coupled ODE system. 

Consequently, calculating the Markovian solutions to these BSDEs is a sim- 
ple matter of evaluating an initial value problem, for which many good numerical 
methods exist. This has applications to calculating risk-averse prices (for exam- 
ple, bid-ask prices) for market models driven by Markov chains. We have also 
seen how this suggests new stochastic methods for the solution of large ODE 
systems. 



12 



References 



[1] Christian Bender and Robert Denk. A forward scheme for backward sdes. 
Stochastic Processes and their Applications, 117(12):1793 - 1812, 2007. 

[2] Bruno Bouchard and Nizar Touzi. Discrete-time approximation and montc 
carlo simulation of backward stochastic differential equations. Stochastic 
Processes and their Applications, 111:175-206, 2004. 

[3] Peter Carr, Helyette Geman, Dilip B Madan, and Marc Yor. From local 
volatility to local levy models. Quantitative Finance, 4(5):581-588, 2004. 

[4] Samuel N. Cohen and Robert J. Elliott. Solutions of backward stochastic 
differential equations on Markov chains. Communications on Stochastic 
Analysis, 2(2):251~262, August 2008. 

[5] Samuel N. Cohen and Robert J. Elliott. Comparisons for backward stochas- 
tic differential equations on Markov chains and related no-arbitrage condi- 
tions. The Annals of Applied Probability, 20(1):267-311, 2010. 

[6] Nicole El Karoui, Shige Peng, and M.C. Quenez. Backward stochastic 
differential equations in finance. Mathematical Finance, 7(1):1— 71, January 
1997. 

[7] FA Longstaff and ES Schwartz. Valuing american options by simulation: a 
simple least-squares approach. Review of Financial Studies, 14(1):113-147, 
2001. 

[8] Dilip Madan, Martijn Pistorius, and Wim Schoutens. The valuation of 
structured products using markov chain models. University of Mary- 
land Working Paper, available at http://www.ssrn. com/abstract=1563500, 
2010. 

[9] E. Pardoux and Shige Peng. Adapted solution of a backward stochastic 
differential equation. Systems & Control Letters, 14:55-61, 1990. 

[10] Shige Peng. A generalized dynamic programming principle and Hamilton- 
Jacobi-Bellman equation. Stochastics and Stochastics Reports, 38:119-134, 
1992. 

[11] Jiongmin Yong and Xun Yu Zhou. Stochastic Controls. Hamiltonian Sys- 
tems and HJB Equations,. Springer, Berlin-Heidelberg-New York, 1999. 



13 



Stock price ($) 



Figure 3: Ask price surface for a digital knockout option under minmaxvar, 
7 = 0.1. 




Figure 4: Time t — bid (high) and ask (low) prices for a digital knockout 
option under minmaxvar, 7 = 0.1. 
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